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Second Special Report 


On 18 December the Science and Technology Committee published its Second Report of 
Session 2009-10, Evidence Check 1: Early Literacy Interventions [HC 44]. On 16 February 
2010 the Committee received a memorandum from the Government which contained a 
response to the Report. The memorandum is published as an Appendix to this Report. 


Appendix: Government response 


Every Child a Reader: Reading Recovery 


1. The Government's policy that literacy interventions should take place early on in 
formal education is in line with the evidence. (Paragraph 25) 


2. The Government's position that early literacy interventions are an investment that 
saves money in the long run is evidence-based. (Paragraph 32) 


We welcome the report’s findings and are glad the committee are in agreement with us on 
these points. 


3. Ms Willis is right to acknowledge the need to compare Reading Recovery with 
alternative interventions. We conclude that, whilst there was evidence to support early 
intervention, the Government should not have reached the point of a national roll-out 
of Reading Recovery without making cost-benefit comparisons with other 
interventions. (Paragraph 37) 


Firstly, it is important to emphasise that Reading Recovery is just one part of the three- 
wave Every Child a Reader Programme roll out. 


We believe that the evidence base on ECAR and Reading Recovery was strong enough to 
justify implementing a large-scale intervention programme which incorporates intensive 
specialist tuition for the lowest attaining pupils with a wider commitment to improving 
teaching throughout the school. 


The evidence base at the time the decision was taken included: 


e Research into the first year of ECaR, commissioned by the Institute of Education 
(Burroughs-Lange, 2006). 


e Management information about ECaR which provides pre-and post test results for 
children who receive the intervention, and qualitative insight into schools’ and 
pupils’ views (ECACT, 2006). 


¢ A long history of research into the effectiveness of Reading Recovery, the core 
intensive intervention of ECaR. 
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e In-house analysis of school attainment in ECaR schools, compared to other 
schools. 


e An economic assessment of the return on investment of early intervention to 
address literacy difficulties (KPMG, 2006). 


e Brooks (2002; revised edition 2007) concluded that Reading Recovery is an 
effective intervention, with effect sizes from different studies standing up well in 
comparison to a range of other interventions. 


Another important basis for adopting the Reading Recovery programme was one of 
scalability - currently in the UK, Reading Recovery provides the only national network 
which was capable of training sufficient numbers of teachers to deliver on the scale 
required. Together with evidence of effectiveness, the issue of scalability meant that 
Reading Recovery was accepted as part of the ECAR programme. 


In the case of complex social policy issues, it is rare to find an evidence base which clearly 
indicates exactly how problems should be tackled. In such cases, it is imperative that 
Government acts on the best available evidence in order to begin to tackle entrenched 
problems such as poor literacy, and evaluates the results. 


4. We are concerned by the low quality of data collection in UK trials on literacy 
interventions. Government-funded trials should seek the best data so as to make the 
results as powerful as possible. Running trials that do not collect the best data is a 
failure both in terms of the methodological approach, but also value for money. 
(Paragraph 40) 


As a Department, we can clearly only comment on our own studies and not more generally 
on UK trials on literacy interventions - although we are doing more to encourage 
independent researchers to strengthen their assessments of cost effectiveness. 


We are committed to collecting robust data on our pilots and interventions that is of 
sufficient quality to inform policy decisions. The Department carefully chooses the 
appropriate methodology for evaluating its policies and pilots, based on advice from expert 
social researchers on scientific credibility and practicality in each case, and through 
discussions at its Research Approvals Committee (RAC) and Policy Evaluation Group 
(PEG). The Department is increasing the involvement of expert researchers at an early 
stage in pilot development to help ensure that the methodological data needs drive the 
design of the pilot. More generally, the analytical community work on behalf of the Chief 
Scientific Adviser to ensure that evidence underpins policy development decisions across 
the Department. 


We believe that research involving carefully matched comparison groups can give good 
quality evidence, especially when there are so many variables affecting social policy. When 
deciding on the methodology for evaluating pilots/trials, we need to take account of the 
relative burden of data collection, the cost, and the timescales involved as well as the quality 
of the data. | 


5. The Government should be careful when selecting evidence in support of educational 
programmes that have changed over time. Reading Recovery today differs from its 
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1980s and 1990s ancestors. Evidence used to support a national rollout of Reading 
Recovery should be up-to-date and relevant to the UK. The Government's decision to 
roll out Reading Recovery nationally is not based on the best quality, sound evidence. 
(Paragraph 45) 


We agree with the Committee that research findings need to be interpreted carefully to 
reflect both the time period and the country in which the study has taken place. However, 
they can still give useful evidence on the potential effectiveness of interventions. ECAR 
was piloted in the UK in advance of decisions about roll-out. 


It is important to again emphasise that Reading Recovery is just one element of ECAR. 


The choice of Reading Recovery as the one-to-one intervention wave of the ECAR 
programme was made during the pilot phase led by the Every Child a Chance Trust. 
Information and evidence collected from the pilot was of high quality, and as noted in our 
response to point 3 (above), we felt we had a range of good and relevant evidence to 
support the roll-out decision. 


6. We recommend that the Government should draw up a set of criteria on which it 
decides whether a research project should be a randomised controlled trial. (Paragraph 
49) 


The Magenta Book provides guidance to Government officials undertaking evaluations. It 
is currently being revised to become the single source of guidance on evaluation for 
analysts and policy makers in government (see response to point 15). There are several 
different types of evaluations and methods for conducting evaluations, including 
Randomised Controlled Trials (RCTs). The revised Magenta Book will outline the pros 
and cons of RCTs and other methods for addressing different types of research questions. 
It will also provide analysts and policy developers with criteria to determine the 
appropriate evaluation method in a given scenario, and offer guidance on how to conduct 
the evaluation. 


7. We conclude that a randomised controlled trial of Reading Recovery was both 
feasible and necessary. (Paragraph 54) 


A Randomised Controlled Trial (RCT) of ECAR, which is a school level programme, 
would have required random allocation of schools to the programme. This would have 
been very complex in the circumstances, given the need for teacher training to be planned 
and supported at local level. 


A RCT of Reading Recovery itself was not considered good value because there is already a 
strong evidence base on the effectiveness of the programme, including international RCTs. 
It is also the case that research involving carefully matched comparison groups can give 
evidence of an acceptable quality, especially when in many areas of social policy there are 
so many different variables which affect the outcome. Well-constructed quasi- 
experimental studies can therefore provide a fit-for-purpose solution, and the evidence 
gained from the ECAR pilot was robust. 
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8. We recommend that the Government identify some promising alternatives to 
Reading Recovery and commission a large randomised controlled trial to identify the 
most effective and cost-effective early literacy intervention. (Paragraph 55) 


There are a very large number of initiatives currently available commercially. It would not 
be feasible to include all of them, in one evaluation. Instead, reviews such as those 
undertaken by Brooks compare the impacts of different initiatives across different studies. 


We have already committed to regularly updating’ “What works for Children with Literacy 
Difficulties’ (Brooks, 2007) as part of our on-line guidance, a recommendation from the 
report ‘Identifying and Teaching Children and Young People with Dyslexia and literacy 
difficulties’ (Rose, 2009). 


Also, as the Committee is aware, DCSF has commissioned an independent evaluation of 
Every Child a Reader, which is running from autumn 2009 to March 2011. The evaluation 
is using a mixed-method approach to evaluate how the programme has been implemented, 
its impact on outcomes, and its value for money. We will be keeping the evaluation 
process and the outcomes under review. 


9. Teaching children to read is one of the most important things the State does. The 
Government has accepted Sir Jim Rose's recommendation that systematic phonics 
should be at the heart of the Government's strategy for teaching children to read. This 
is in conflict with the continuing practice of word memorisation and other teaching 
practices from the 'whole language theory of reading’ used particularly in Wave 3 
Reading Recovery. The Government should vigorously review these practices with the 
objective of ensuring that Reading Recovery complies with its policy. (Paragraph 59) 


The term ‘whole language theory of reading’ has acquired a pejorative meaning, and is used 
to imply that a theory or methodology has no regard for the use of print information, 
sound/letter learning, phonics or the ability to decode text, and that children are 
encouraged to invent or guess text. This is not an accurate description of Reading 
Recovery. The reference to ‘the practice of word memorisation' in Reading Recovery is 
equally inappropriate. Reading Recovery teachers are given very explicit instructions to the 
contrary. 


Memorisation is not an alternative to decoding in Reading Recovery. However, the Rose 
review of early literacy clearly states that ability to decode is not the end point of literacy 
learning: ‘It is widely agreed that phonic work is an essential part, but not the whole 
picture, of what it takes to become a fluent reader and skilled writer, well capable of 
comprehending and composing text.’ (Rose, 2006, paragraph 37) 


Decoding is an essential skill for a reader encountering new, unexpected or unfamiliar 
words, but an efficient, fluent reader must develop a system whereby words read frequently 
can be read automatically. However, it is evident from the research literature that the 
balance of learning needs across the two dimensions changes as children become more 
fluent and automatic readers of words. Most readers learn intuitively how to retain in the 
memory that which they have deciphered, but for very low attaining children in Reading 
Recovery this skill may have to be taught explicitly. This is entirely different from the use 
of memorisation to circumvent the development of decoding skills. 
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Dyslexia 


10. The Rose Report's definition of dyslexia is exceedingly broad and says that dyslexia 
is a continuum with no clear cut-off points. The definition is so broad and blurred at 
the edges that it is difficult to see how it could be useful in any diagnostic sense. 
(Paragraph 71) 


We recognise that views on identifying and supporting children with dyslexia vary. 


The Rose report reviewed the many published definitions of dyslexia and the Government 
has accepted the report’s definition which proposes that dyslexic difficulties are best 
thought of as existing on a continuum of wider literacy difficulties, rather than forming a 
discrete category. Dyslexia is not unique in this regard and a number of other conditions, 
such as autism spectrum disorder, are on a continuum. 


We believe it is useful to have a working definition of dyslexia so that, as the Rose report 
concluded, “we can build professional expertise in identifying dyslexia and developing 
effective ways to help learners overcome its effects.” 


11. We conclude that 'specialist dyslexia teachers' could be renamed ‘specialist literacy 
difficulty teachers'. There are a range of reasons why people may struggle to learn to 
read and the Government's focus on dyslexia risks obscuring the broader problem. The 
Government's support for training teachers to become better at helping poor readers is 
welcome and to be supported, but its specific focus on ‘specialist dyslexia teachers’ is 
not evidence-based. (Paragraph 77) 


The Government accepts Sir Jim Rose’s view that: “There is also a need to develop better 
access for schools, parents and children to the advice and skills of specialist dyslexia 
teachers, who can devise tailored interventions for children struggling most with literacy, 
whether or not they have been identified as having dyslexia.” 


The term “specialist dyslexia teacher” is widely used and the British Dyslexia Association 
accredits courses meeting its professional criteria. Dyslexia difficulties are often not 
confined to literacy difficulties. Pupils with dyslexia often have co-occurring difficulties 
which may be seen in aspects of language, motor co-ordination, mental calculation, 
concentration and personal organisation. Given these points, we believe it is sensible to 
continue with the “specialist dyslexia teacher” term rather than introduce an additional 
one. 


However, we do accept that accreditation needs to reflect the best evidence-based practice, 
which will change over time. We have worked with the British Dyslexia Association to 
review their accreditation criteria of courses for specialist dyslexia teachers. 


12. We recommend that future research on the impact of literacy interventions on 
children with dyslexia should be well designed randomised controlled trials, using 
appropriate control groups (including children with other reading difficulties and 
‘normal' children), and test a range of literacy interventions. (Paragraph 82) 


The Government recognises that more good quality evidence is required around the 
effectiveness of particular interventions for children with dyslexia. In addition, the 
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department recognises that Randomised Controlled Trials (RCTs) are an important 
research tool and is committed to carrying them out where appropriate. 


However, it is important to choose the most appropriate methodology. For a number of 
reasons, including ethical, funding, recruitment and timing issues, RCTs are not always 
possible or advisable in every circumstance. The Department, therefore, carefully 
considers all factors before commissioning research that will most usefully add to the 
existing evidence base. 


In addition, the Department will take on board results of other research that has been 
carefully scrutinised and deemed as high quality when making policy and funding 
decisions related to interventions for dyslexia. 


We will be asking Ofsted to undertake a survey to evaluate the extent to which, and with 
what impact, primary and secondary schools are using interventions to advance the 
progress of children and young people experiencing a wide range of literacy difficulties. It 
will be timed to provide an opportunity to evaluate the implementation of Jim Rose’s 
recommendations. 


13. We recommend that the Government be more independently minded: it should 
prioritise its efforts on the basis of research, rather than commissioning research on the 
basis of the priorities of lobby groups. (Paragraph 84) 


The Government has set out its priority to improve the educational outcomes for children 
with special educational needs and has commissioned its own research. We will be 
publishing shortly "Special Educational Needs and Disability: Understanding Local 
Variation in Prevalence, Service Provision and Support". 


Expert stakeholder groups have a role to play in the development of policy and research 
priorities; indeed some stakeholder groups actively commission high quality research 
which helps inform debate. The ultimate decisions on its research and policy priorities lie 
with the Government. 


Conclusions 


14. In broad conclusion, we found that there was a willingness from the Department to 
base its approach to early literacy interventions on the evidence. However, we 
discovered worryingly low expectations regarding the quality of evidence required to 
demonstrate the relative effectiveness and, in particular, the cost-effectiveness of 
different programmes. (Paragraph 87) 


We believe that when implementing ECAR, the decision to roll out nationally was made on 
the basis of sound evidence that the programme was effective, coupled with the need to 
deliver a stable intervention programme. | 


We absolutely recognise the importance of strengthening analysis of cost-effectiveness of 
different policies in the future. We can reassure the Committee that we have built this into 
our research programme and now consider whether a stronger focus on cost effectiveness 
is required in all our research projects as part of our Research Approvals Committee. 
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Furthermore the Department is doing more to encourage academics and other researchers 
to build in a cost-effectiveness component into all their (externally funded) research work, 
and to avoid focusing only on impact/effect size. 


15. We recommend that the Government review its Magenta Book with a view to 
raising its expectations of social science research and evidence in relation to policy. 
(Paragraph 88) 


The Magenta Book is already under review. A revision was commissioned by the 
Government Social Research Unit in Autumn 2009 and was awarded to the research 
consultancy SQW. A revised version will be available later in 2010. The aim of the revision 
is to edit and develop the Magenta Book to increase its influence and utility, and become 
the single source of government guidance on evaluation for analysts and policy makers. 


The review of the Magenta Book is part of a programme of work to strengthen government 
capacity and skill in evaluation. This is being led by the Cross Government Evaluation 
Group (CGEG), set up last year in response to a Government Economic Service Initiative. 
The remit of this cross-disciplinary group is to: 


a) Identify and disseminate the conditions necessary for successful evaluation in 
government. 


b) Strengthen guidance on evaluation and raise its status. 
c) Strengthen training on evaluation and raise its status. 


d) Identify ways to increase the perceived value of evaluation amongst policy and delivery 
colleagues in their decision-making, including advising on when and what to evaluate. 


e) Support policy and delivery colleagues in their use of evaluation findings. 


f) Develop internal evaluation capacity through facilitating cross-departmental 
knowledge sharing, learning and development and peer review. 


g) Strengthen links with academics, evaluation experts, and relevant stakeholders both 
nationally and internationally, building international good practice. 


h) Identify and address cross cutting policy, programme and operational evaluation 
issues. 


i) Clarify the role and value of evaluation’ in relation to the range of other commitments 
such as post-implementation monitoring, audits and reviews carried out by the NAO, 
PMDU, etc. Where possible, try to strengthen links between the various requirements 
and those responsible for them (e.g. those responsible for evaluation and those 
responsible for spending reviews). 


j) Report main findings to Heads of Analysis (and Government chief economist where 
necessary) and encourage them to influence top level stakeholders. 
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